Papers with open-source LLMs
LEAF: Learning and Evaluation Augmented by Fact-Checking to Improve Factualness in Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) struggle with factual accuracy in knowledge-intensive domains like healthcare. |
| Approach: | They propose a framework for improving LLM factuality in medical question answering . RAFE, Fact-Check-then-RAG and Learning from Fact Check are components . |
| Outcome: | Experimental results show that LEAF outperforms Factcheck-GPT in detecting inaccuracies and corrects errors without labeling . the framework provides a scalable solution for industrial applications requiring high factuality scores. |
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing instruction-tuned open-source LLMs have only been instruction- tuned for English and a few popular languages, thus hindering their accessibility to many other languages in the world. |
| Approach: | They propose a framework that uses supervised fine-tuning and reinforcement learning from human feedback to improve the accessibility of large language models. |
| Outcome: | The proposed framework enables the evaluation of generative LLMs in multiple languages. |
Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety (2026.acl-long)
Copied to clipboard
Can Jin, Rui Wu, Tong Che, Qixin Zhang, Hongwu Peng, Jiahui Zhao, Zhenting Wang, Wenqi Wei, Ligong Han, Zhao Zhang, Yuan Cao, Ruixiang Tang, Dimitris N. Metaxas
| Challenge: | OpenAI introduces deliberative alignment (DA) to enhance safety of its o-series models, but effectiveness of this approach in open-source LLMs is understudied. |
| Approach: | They propose a case-augmented deliberative alignment method for large language models . they propose to use reinforcement learning on self-generated safety reasoning chains . |
| Outcome: | The proposed method avoids narrowly enumerated rules and allows broader adaptability. |
More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)
Copied to clipboard
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
| Challenge: | Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance? |
| Approach: | They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions. |
| Outcome: | The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset. |
Divide, Link, and Conquer: Recall-oriented Schema Linking for NL-to-SQL via Question Decomposition (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Open-source LLMs often depend on large proprietary models, which introduce serious privacy concerns. |
| Approach: | They propose a plug-and-play framework that improves SQL generation for smaller LLMs . they propose to apply question decomposition at the schema linking stage rather than during SQL generation . |
| Outcome: | The proposed framework improves schema linking recall by 25.1% and execution accuracy by 8.2% on the BIRD benchmark. |
TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved the functional correctness of code translation, but execution efficiency remains overlooked. |
| Approach: | They propose a benchmark to explicitly assess execution efficiency in LLM-translated code. |
| Outcome: | The proposed benchmark identifies that execution efficiency is an essential dimension of code translation . the results highlight that correctness and efficiency are often misaligned . |
MuMath: Multi-perspective Data Augmentation for Mathematical Reasoning in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) that integrate with external Python interpreters are not able to demonstrate the calculation process, which compromises user-friendliness and understanding of problem-solving steps. |
| Approach: | They propose to use LLaMA-2 to refine LLti-perspective augmentation methods to improve performance. |
| Outcome: | The proposed model achieves 88.3% on GSM8K and 34.5% on MATH. |
SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced rapidly from conversational problem solving to addressing real-world tasks involving tool use, such as software engineering (SWE). |
| Approach: | They propose to build an LLM-based software engineering agent that synthesizes test cases and scales up agent trajectories to build training data. |
| Outcome: | The proposed model outperforms state-of-the-art models on the SWE-bench-Verified benchmark. |
Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to adapt pretrained Large Language Models to new lower-resource languages are limited to English. |
| Approach: | They propose to combine cross-lingual instruction-tuning with additional monolingual pretraining to adapt LLMs to new lower-resource languages. |
| Outcome: | The proposed model is the first open-source instruction-following LLM for Estonian . the proposed model improves commonsense reasoning and multi-turn conversation capabilities . |
MuMath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | a method to combine the advantages of open-source and tool-free LLMs remains to be explored. |
| Approach: | They propose a method to integrate open-source LLMs with external Python interpreters and augment math reasoning data. |
| Outcome: | The proposed method improves on GSM8K and MATH with the use of external tools. |
Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research focuses on developing powerful large language models for mathematical reasoning within monolingual languages. |
| Approach: | They propose to use translation to build powerful multilingual math reasoning models . they propose different training strategies to build xMR LLMs that outperform open-source LLM . |
| Outcome: | The proposed model outperforms open-source LLMs and surpasses ChatGPT in few-shot scenarios. |
Matina: A Large-Scale 73B Token Persian Text Corpus (2025.naacl-long)
Copied to clipboard
Sara Bourbour Hosseinbeigi, Fatemeh Taherinezhad, Heshaam Faili, Hamed Baghbani, Fatemeh Nadi, Mostafa Amiri
| Challenge: | Existing Persian datasets are small and lack content diversity . lack of high-quality data has slowed development of NLP models and open-source LLMs for Persian. |
| Approach: | They propose a Persian dataset of 72.9B tokens that is preprocessed and deduplicated to ensure high data quality. |
| Outcome: | The proposed model performs well on key Persian NLP tasks. |
Walking in Others’ Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing prompting methods that require white-box access to the model or substantial training fail to simultaneously lessen toxicity and bias. |
| Approach: | They propose a strategy that encourages LLMs to integrate diverse human perspectives and self-regulate their responses by incorporating diverse human viewpoints. |
| Outcome: | The proposed approach can significantly diminish toxicity (up to 89%) and bias (up 73%) in LLMs’ responses. |
Can Large Language Models perform Relation-based Argument Mining? (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for RbAM fail to perform satisfactorily across different datasets. |
| Approach: | They propose to use relation-based argument mining to determine agreement (support) and disagreement (attack) relations amongst textual arguments in binary and ternary settings. |
| Outcome: | The proposed method outperforms the best performing (RoBERTa-based) baseline on two open-source LLMs and with GPT-3.5-turbo on several datasets for (binary and ternary) RbAM. |
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal. |
| Approach: | They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages. |
| Outcome: | The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark. |
mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models show low performance for lesser resourced languages, but they can achieve surprising performance on complex reasoning tasks in natural language processing (NLP). |
| Approach: | They compile the first large-scale multilingual math reasoning dataset, *mCoT-MATH*, covering eleven diverse languages. |
| Outcome: | The proposed model achieves impressive consistency across languages and comparable performance to close- and open-source models even of much larger sizes. |
LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have been proposed as general-purpose agents for experimental design . eval: LLMs show no sensitivity to experimental feedback. |
| Approach: | They propose a method that combines LLM prior knowledge with nearest-neighbor sampling to guide the design of experiments. |
| Outcome: | The proposed method outperforms classical methods in the design of experiments. |
CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems (2024.findings-acl)
Copied to clipboard
| Challenge: | a number of studies have evaluated user satisfaction estimation in TOD systems . current benchmarks for user satisfaction estimates are highly skewed towards dialogues for which the user is satisfied. |
| Approach: | They leverage large language models to generate satisfaction-aware counterfactual dialogues to augment original dialogues of a test collection. |
| Outcome: | The proposed models show higher robustness to increase in dissatisfaction labels than fine-tuned models. |
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM (2023.findings-emnlp)
Copied to clipboard
Sahal Mullappilly, Abdelrahman Shaker, Omkar Thawakar, Hisham Cholakkal, Rao Anwer, Salman Khan, Fahad Khan
| Challenge: | Recent large language models like ChatGPT and Bard excel in a wide variety of NLP tasks but are not specifically tailored for climate related domain specific information. |
| Approach: | They propose a lightweight Arabic Mini-ClimateGPT that is built on an open-source LLM and specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct. |
| Outcome: | The proposed model surpasses the baseline LLM in 88.3% of cases during ChatGPT-based evaluation and human expert prefers it over other open-source models. |
Can LLMs be Literary Companions?: Analysing LLMs on Bengali Figures of Speech Identification (2025.emnlp-main)
Copied to clipboard
| Challenge: | despite Bengali being among the most spoken languages, the NLP efforts on it remain limited. |
| Approach: | They present a dataset that includes Bengali figures of speech on six poets . they deploy state-of-the-art Large Language Models to the dataset and fine-tune the best models . |
| Outcome: | The proposed dataset reveals that two open-source LLMs perform better than others in Bengali . the framework can be reproduced for English and other low-resource languages . |
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not noticed the safety risks of large language models . authors evaluated 1,400 questions in multi-turn dialogue coreference . |
| Approach: | They are the first to evaluate LLM safety in multi-turn dialogue coreference . they created a dataset of 1,400 questions and tested five open-source models . |
| Outcome: | The study shows that model safety decreases in multi-turn dialogue coreference scenarios . the highest success rate was with the LLaMA2-Chat-7b model, while the lowest was with mistral-7B-Instruct model . |
UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking (2025.findings-emnlp)
Copied to clipboard
Sarfraz Ahmad, Hasan Iqbal, Momina Ahsan, Numaan Naeem, Muhammad Ahsan Riaz Khan, Arham Riaz, Muhammad Arslan Manzoor, Yuxia Wang, Preslav Nakov
| Challenge: | Existing automated fact-checking systems are predominantly developed for English . Existing systems focus on claim verification, but UrduFactQA targets factuality . |
| Approach: | They propose two hand-annotated benchmarks to enable fact-checking and factual consistency evaluation in Urdu. |
| Outcome: | The proposed benchmarks are the first of their kind for Urdu and are available online. |